Automatic registration and fusion of single/multiple imaging modalities is beneficial for various diagnostic and interventional procedures. Specifically, the registration of low-cost, real-time imaging such as ultrasound (“US”) with prior three-dimensional (“3D”) imaging (e.g., computed tomography “CT”) or magnetic resonance imaging (“MRI”) is desirable. To this end, numerous image registration techniques have been proposed. Some known techniques take advantage of the correlation between intensity values of corresponding content (e.g., an anatomical object) in both sides of the registration (i.e., non-segmentation-based image registration), while other known techniques, due to lack of such correlation, are anatomy-segmented driven, whereby the anatomy of interest is first segmented in both images and the registration is then conducted between the segmented objects using shape information and/or matching labels (i.e., segmentation-based image registration). The assessment of the registration and/or segmentation accuracy is always a challenge since a ground truth for the registration and/or segmentation doesn't exist.